Statistical clustering techniques in historical English linguistics

ed 1 0 1 0 0 0 0 ... ... ... ... ... ... ... ... The information in Table 2 can be used to measure similarity between adjacent periods: Each period is represented not by a single frequency value but in fact by a vector of frequency values for the past participles; relative similarity between adjacent periods can be computed with a statistic such as Pearson's correlation r. The first step of VNC is to find the two periods that correlate best, i.e., whose correlation coefficient is highest. In this case, periods two (1700-49) and three (1750-99) exhibit the highest correlation. The correlation between their collocate sets is shown in the left panel of Figure 4, in which it can be seen that especially the more frequent past participles correlate rather well. These two periods are then merged, i.e. a new period is created that holds the respective mean values for each past participle (1 for abandoned, 2 for abated, 1 for abolished, etc.). The dendrogram in the right panel of Figure 4 results from successive mergers of this kind. The dendrogram allows several ways to partition the ARCHER data, but more importantly, it also makes clear that certain default periodizations would be problematic. For instance, cutting up the data into century-length chunks would obscure the relatively greater differences that are found in the 1800s and 1900s. By contrast, if the data were to be used for a binary contrast of before and after, it would make sense to split the data by the year 1900. Especially for a distinctive collexeme analysis, that would be a reasonable option to bring out collocational differences in the development of the English passive. 6. Detection of outliers with Variability-based Neighbor Clustering The final application of VNC to be mentioned here is the detection of outliers in fine-grained historical data. This is particularly useful in the analysis of year-by-year data that can exhibit substantial fluctuation. For example, Gries & Hilpert (2010) study the decline of the third person singular -(e)th in English. Between the early 15th century and the late 17th century, forms such as